Improving Direct Search algorithms by multilevel optimization techniques

Direct Search algorithms are classical derivative-free methods for optimization. Though endowed with solid theoretical properties, they are not well suited for large-scale problems due to slow convergence and scaling issues. In this paper, we discuss how such limitations can be circumvented, on problems for which a hierarchy of objective functions is available, by using multilevel schemes which are able to accelerate the computation of a finest level solution. Starting from a previously introduced derivative-free multilevel method, based on Coordinate Search optimization with a sampling strategy of Gauss-Seidel type, we consider also the use of sampling strategies of Jacobi type, and present several algorithmic variations which yield more accurate solutions. Experiments performed on two model problems justify our choices, and show that accurate solutions can be obtained in a reasonable time, even in the case of large-scale instances.